Papers with real world scenarios
Memorization vs. Generalization : Quantifying Data Leakage in NLP Performance Evaluation (2021.eacl-main)
Copied to clipboard
| Challenge: | Public datasets are often used to evaluate the efficacy and generalizability of state-of-the-art methods for many tasks in natural language processing (NLP). |
| Approach: | They identify leakage of training data into test data on several publicly available datasets used to evaluate NLP tasks, including named entity recognition and relation extraction. |
| Outcome: | The proposed model can memorize and generalize data on several publicly available datasets and is compared against previously unseen data. |